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Abstract: This paper presents the face recognition based on Enhanced GABOR LBP and PCA. Some 
of the challenges in face recognition are occlusion, pose and illumination .In this paper, we are more 
focused on varying pose and illumination. We divided this algorithm into five stages. First stage finds 
the fiducial points on face using Gabor filter bank as this filter is well known for illumination 
compensation. Second stage applies the morphological techniques for reduce useless fiducial points. 
Third stage applies the LBP on reduced fiducial points with neighborhood pixel for improving the pose 
variation. Forth stage uses PCA to detect the best variance points which are necessary to characterize 
the training images. The last recognition stage includes finding the Euclidean norm of the feature 
weight vectors with the test weight vector. In this project, we used 20 images of 20 different persons 
from ORL database for training. For testing, we used images with varying illumination, pose and 
occluded images of the same training persons. Using this algorithm, testing results has shown 
significant improvement performance. 
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I. Introduction 

Face recognition is a very active research topic in the field of pattern recognition and computer 
vision[l]. Face recognition systems are important systems that can be used in various application areas such as 
entertainment, smartcards, information security, and law enforcement and surveillance. From a problems point 
of view, there are five major problems in face recognition which affect performance of a system: 1) Illumination 
variations 2) Pose changes 3) expression variations 4) time delay 5) Occlusions. Algorithms presented for pose 
variation are divided in two main categories based on their type of gallery images [2]. First of all, Multi-view 
face recognition system (FRS)[3] that requires several poses for every subject in gallery. Another category 
identifies probe faces that has different poses with gallery face. In general situations, a frontal face in gallery is 
available, but probe faces have unpredicted pose variations. So, the system must be robust for these types of 
situations. A key issue in face recognition is to find effective descriptors for face appearance. Hence it is 
difficult to develop a system which can recognize human faces under variable conditions and environment. 

Face recognition has drawn a great deal of attention from the researchers and industrial communities 
over the past several decades. Since the early 70 's, face recognition has drawn the attention of researchers in 
various fields, which include security, psychology, image processing and computer vision. Research in 
automatic face recognition started in the 1960s. Numerous approaches have been proposed, including eigenfaces 
[4], laplacianfaces [5], neural networks [6], elastic bunch graph matching [7], to overcome the challenges for 
recognition. To recognize face under illumination and pose variation of face image we introduced an image 
computationally the first method came into existence was template based matching. It required the administrator 
to locate the feature such as eyes, nose, and mouth on the photographs before it calculated distance and ratios 
between common reference points. This initial method had poor performance as the results were calculated 
manually. 

Holistic methods tend to blur out small details owing to residual spatial registration errors. On the other 
hand, local descriptor methods have drawn increasing attention because they can capture small appearance 
details. Local descriptor methods include Gabor wavelets [8], Local Binary Patterns (LBP) [9], Local Ternate 
Patterns (LTP) [10]. Gabor wavelets were successfully used in face recognition. Moreover this method was not 
adaptable in any kind of variations. The approaches afterwards can be classified into two categories; including 
subspace based holistic feature and local appearance feature which brought a new revolution in the field of face 
recognition. In holistic matching method, the whole face image is represented as a high -dimensional vector. Due 
to the size of dimensionality, such vectors cannot be compared directly. Hence, holistic methods use 
dimensionality reduction techniques to resolve this problem and thus derive lower -dimensional vectors for 
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subsequent classification. The most popular examples among such approaches are based on Principal 
Component Analysis (PCA) [11] and on Linear Discriminant Analysis (LDA) [12] Independent Component 
Analysis(ICA)[13]. 

The LBP is a powerful illumination invariant texture primitive. The histogram of the binary patterns 
computed over a region is used for texture description. LTP is a generalization of LBP. The Local Binary 
Pattern is originally proposed by Ojala for the aim of texture classification, and then extended for various fields, 
including face recognition, face detection [14], facial expression recognition. The Local Binary Pattern is a non 
parametric operator which is used for describing a local spatial structure of an image. Face image is divided into 
several regions and LBP is applied and features are extracted over the region. These Features are concatenated 
to form face descriptor. Although face recognition with local binary pattern has been Proven to be a robust 
algorithm, it suffers from heavy Computational load due to the very high dimensional feature vectors that are 
extracted by concatenating the LBP histograms from each local region [24]. 

In 1988, Kirby and Sirovich applied principal component analysis , a standard linear algebra technique, 
used to transform one set of variables into another smaller set, and the newly created variables are not usually 
easy to interpret. ICA is a generalization of PCA, which is sensitive to high order relationship among the image 
pixels. The primary advantages of these techniques are that it provides more robust to the effect of noise. The 
local approaches, such as Gabor filter extract information from local facial features to distinguish faces, and 
have the advantage of robustness to environmental changes like illumination and expressional variability, so are 
used extensively in face recognition. The local binary pattern (LBP) features are originally designed for texture 
description. The operator has been successfully applied to pose change and facial expression analysis and face 
recognition. 

The idea behind using the LBP features is that a face can be seen as a composition of micro patterns. 
LBP in nature represents the first-order circular derivative pattern of images, a micro pattern generated by the 
concatenation of the binary gradient directions. However, the first-order pattern fails to extract more detailed 
information contained in the input object. To the best of our knowledge, no high-order local pattern operator has 
been investigated for face representation. In fact, the high-order operator can capture more detailed 
discriminative information. Some high-order non local pattern methods have been successfully used to solve the 
face recognition problem. 

We work on face recognition with combining the holistic, local approach and LBP. Gabor wavelets 
capture the local structure corresponding to specific spatial frequency, spatial locality, and selective orientation 
which are demonstrated to be discriminative and robust to illumination and expression changes. LBP operator 
which describes the neighboring changes around the central point is a simple yet effective way to represent 
faces. In this paper, these combinations of LBP and Gabor features have improved the face recognition 
performance significantly compared to the individual representation. The feature points extracted using Gabor 
Filter and LBP on feature point that shows better response irrespective of the illumination, rotation and 
occlusion. The performances of three statistical face recognition techniques namely Principal Component 
Analysis (PCA) Combined LBP Gabor and PCA for selecting fiducial points gives better recognition result. 

Facial feature point extraction is explained overview of Gabor Filter, LBP and PCA for better 
recognition. Last sections will give the comparison of experimental results and compare all the results with 
conclusion that how it improves illumination and pose variation recognition. 

II. Enhanced Gabor Feature Extraction 

Gabor filters are used in representation of face due to their biological relevance and computational 
properties. Gabor filters are principally band pass filters which exhibit desirable characteristics of spatial 
localization and orientation selectivity, spatial frequency selectivity and quadrature phase relationship. Gabor 
filter representations of face image are robust for illumination and expressional variability, so are used 
extensively in face recognition. Therefore, when the goal is the derivation of local and discriminating feature, 
adopting Gabor transformation has become a popular method of feature extraction. In our work, 2D Gabor 
wavelet is applied to create are presentation of facial features. The Gabor kernels can be defined as following: 



(z) = \KA c -f||k u . v || 2 ||z|| 2 |2a 2 ) 



gik|j jV z 



a) 



Where u and v define the orientation and scale of the Gabor kernels, z=(x, y),||. || denotes the norm operator, 
and thewave vector k„ v is defined as follows: 

k M , v = k v e i4 V (2) 

Where k v = ^and <D M = — .f is the spacing factor between kernels in the frequency domain. The Gabor Kernel 

is a product of Gaussian envelope which is shown outside the square brackets in and a complex plane wave 
which is the first term inside the square brackets determines the oscillatory part of the kernel and the second 
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term compensates for the dc value. The remaining kernels can be generated from the mother wavelet by scaling 
and rotating the mother wavelet. 

The Gabor wavelet representation of an image has been designed image features at different locations having 
different frequency can be extracted by convolving the image I(x, y)with the filters. Let I(x, y) be a gray scale 
image, the convolution of image I and a Gabor kernel QPM is defined as follows 

o„ = '<*) * P„ (*) (3) 

Where z=(x, y)* denotes the convolution operator, and Ofi, vis the convolution result corresponding to the 
Gabor kernel at orientationu and scale v. 

Enhanced Gabor filter is enhancement in Gabor filter with the use of morphological techniques that adjust the 
contrast and clearing the border in an image and label only the useful feature points. These are used for feature 
extraction, texture analysis and texture segmentation, target detection, fractal dimension management, 
Document analysis, edge detection, retina identification, image coding and image representation. The main 
advantage of this approach is that if prominent facial features are occluded, for instance due to facial hair, 
glasses or expressions even then the algorithm should still recognize the face optimally basing on the features 
extracted from other regions of the face. However, they perform poorly when classifying face images having 
even a slight side rotation. 

III. Local Binary Pattern (LBP) 

The LBP operator is one of the best performing texture Descriptors The standard version of the 
LBP of a pixel is formed by thresholding the 3X3 neighborhood of each pixel value with the center pixel's value 
and concatenating the results binomially to form a number. Let Z 0 be the center pixel gray level and Zi (i=0, 1, 
7) be the gray level of each surrounding pixel. Fig. 1 illustrates the basic LBP operation. If Ziis smaller than Z 0 , 
the binary result of the pixel is set to 0 otherwise set to 1 is in equation. The eight binary associated with eight 
neighbors are the read sequentially in clockwise direction to form binary equivalent. This binary equivalent in 
decimal system may be assigned to the central pixel and may be used to characterize to local texture is shown in 
fig 2. 

/(/(2b) s /(£■)) 
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This method makes use of the most frequently occurred patterns to capture descriptive textural 
information. LBP is a grayscale invariant texture measure and is a useful tool to model texture images. 
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Figure 1. Eight neighborhood around central pixel Zo 

An LBP can also be considered as the concatenation of the binary gradient directions, and is called micro 
pattern. When the threshold is set to zero. The histograms of these micropatterns contain information of the 
distribution of the edges, spots, and other local features in an image. LBP has been successfully used for face 
recognition. 




S5 


9i 


21 








57 


12 


13 



Threshold 



1 


1 


0 


I 




1 


1 


0 


0 



Binary: II 0010 II 
Decimal: 203 

Figure 2. Calculating LBP from 3x3 window 
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IV. Principal Component Analysis 

The Principal component analysis (PCA) is often referred to as a technique for reducing the number of 
variables in a data set without loss of information, and as a possible process for identifying new variables with 
greater meaning. Unfortunately, while PCA can be used to transform one set of variables into another smaller 
set, the newly created variables are not usually easy to interpret.PCA has been most successful in applications 
such as image compression. In many applications, PCA is used to reduce the dimensionality of a data set 
consisting of a large number of interrelated variables, while retaining as much as possible of the variation 
present in the data set. The most of the variation PCA identify directions in which there is very little variation. 
The eigenvectors correspond to the directions of principal components of the original data and their statistical 
significance is given by their corresponding eigenvalues. The eigenvectors constitute the transformation matrix. 
Eigen-vectors are unit vectors that identify the basic characteristics of the input data in a nonparametric format 
so eigenvectors are also called basis vectors. The eigenvectors form the basis of the feature space and the 
eigenvalues show the weightings of the individual contributions of the eigenvectors. Since the eigenvectors are 
unit vectors, their contribution incompletely normalized. However, the relative importance of the eigenvectors is 
indicated by the corresponding eigenvalues. The larger the eigenvalue the more the associated eigenvector 
contributes to explaining the variance of the input data. The significant eigenvalues and their associated 
eigenvector called principal components. 

4.1. PCA Algorithm 

As the standard PCA algorithm is well known, we briefly define in the algorithm. In Standard PCA 
consists of two stages, Training stage and Recognition stage. Training stage is used to convert the original space 
to subspace using basis vectors correspond to maximum variance direction in the original space is in fig 3. Let 
p-dimensional space data is transformed to q-dimensional data using the linear transformation. Where q«p 
which is of key importance in case of large database. Let us see how the transformation function is derived and 
face space is formed. Suppose the given database consists of K number of MxN size images. We adjust these 
images column wise vectors of ((MxN)xK) size so that we can calculate the covariance matrix for one 
dimension signal which is of less complexity. We then calculate the mean vector of all images and mean 
adjusted them to ensure that the Eigen vector corresponding to Eigen value represents the dimension in Eigen 
space in which variance of vectors is maximized in correlation sense. 

B=x i -x i (4) 
x — —Hf=o x t (5) 

Xi is column vector of i th image. The covariance matrix is found from the mean subtracted vectors. 

C = BxB T (6) 

The linear transform function can be found using Eigen vectors corresponding to Eigen values (i.e., selected by 
thresholding process) found from the covariance matrix. 

\ { e { = Ce { (7) 

By solving the equation 4 we get Eigen values (A4) and corresponding Eigen vectors (eO. The Eigen values are 
arranged in decreasing order because the largest Eigen values one of more importance as they represent 
maximum variance in the space. We select the Eigen values by thresholding because not all Eigen values are 
necessary to represent the original space and this helps in dimension reduction. The corresponding Eigen vectors 
of selected Eigen values are used to linearly transform the original space to feature space or face space. 
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Figure 3. Principal Component analysis 



The eigenvectors correspond to the directions of principal components of the original data and their statistical 
significance is given by their corresponding Eigenvalues. The database images are then projected onto this 
feature space (the feature space is the collection of parameters) or face space. 

The recognition stage consists of mean adjusted test image projected onto face space. Then Euclidean distance is 
calculated between projected test image and all projected images in the face space. The image satisfying the 
minimum Euclidean distance will be the matched face. Thus the projected test image (noisy image) is gradually 
replaced by the stored image or restoring the image. 
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V. Combined Recognition With Enhanced Gabor LBP And PCA 

In paper, Gabor filter bank is proposed and combined with the local binary pattern. Both the theoretical 
analysis and the experiment results show that the combined LBP Gabor filter is effective for illumination and 
pose variation. In our experiments, we choose 6 orientations and 5 frequencies and 30 Gabor filters are 
combined to form a Gabor filter bank. We are taking the 10 database images for training the recognition system. 
The noise is removed by applying the median filter and Gabor bank is used for extracting the feature points. The 
morphological enhancement is used to correct uneven illumination and reduce the useless feature points. This 
filtering can be used together to enhance contrast and clearing the border in an image and label only the useful 
feature points. Then take centroid of each labeled region of the image. 

The LBP is applied on that features points. It automatically identify the feature points the neighborhood 
pixel of 3x3 windows threshold to the binary equivalent that improves the head rotated face image. In order to 
reduce the dimension further, Principal Component Analysis (PCA) is applied in which each image is 
represented by a t-dimensional feature vector. The PCA can be used to transform the t-dimensional vector 
into a f-dimensional vector, where normally f « t then we can project the test vector to the lower space and 
calculate the minimal Euclidean distances between these vectors. Closest vector is the exact recognized face 
image. 

The recognition process is occur in two stages testing stage having one test face image and training stage. In 
which the training the database of all the training face images and check the Euclidian distance between the test 
and trained image. 

VI. Experiments And Results 

6.1 Experimental Data 

In the experiment we are taking the 10 training set database images as shown in Fig 4. We are testing 
the training database face images at different conditions light, crop, pose and expressions changes. We trained 
the database images with Gabor LBP and PCA algorithm and then test all the face images having different 
conditions is shown in fig 5. The degree of recognition is tested on expression and light gives better results in 
fig:6. 




Figure 4. Sample Images for training set database 




Figure 5. Experimental results of combined Gabor LBP and PCA 




Figure 6.Face expression and light image with Gabor LBP & PCA 
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The comparison results of combined Local binary pattern (LBP) Enhanced Gabor filter and principal component 
analysis (PCA) shows the pose variation improves 79.48% and light effect improves 95.98% so recognition rate 
increases and recognition improves. 

Tablel: Test Results For LBP Enhanced Gabor+PCA and PCA 



Database 


LBP+Enhanced 
Gabor+PCA 


Gabor+PCA 


PCA 


No. of Images 


Light 


95.98 


90 


50 


20 


Crop 


70.25 


65 


50 


20 


Pose variation 


79.48 


55 


46 


10 


Light + Crop 


59 


55 


22.5 


40 


Expression 


85.51 


84.21 


78.95 


19 


Expression+Light 


53 


45 


40 


20 


Expression+Light+Crop 
Pose change 


56 


50 


50 


10 



VII. Conclusion And Future Work 

The Combined performance of Gabor LBP and traditional PCA algorithm gives better result on 
illumination and pose variation. The Gabor filter bank with different frequencies and orientations outputs finds 
the large number of important fiducial facial feature points all over the face. If some portion of the face is 
occluded then the remaining feature points are enough to represent the face. The LBP is applied on Gabor 
feature points and with neighborhood window improves the pose variation recognition result. In this way the 
proposed algorithm outperforms the traditional PCA algorithm in case of occlusion and non uniform pose and 
illumination changes. We also modified the selection criteria for the fiducial points by using morphological 
techniques. This gave good results compare to simple Gabor and PCA in case of combination attacks like 
illumination and occlusion, illumination and expression change. The combined performance improves efficient 
of face recognition but poor performance for large pose variation or orientation change. 
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